Joint Inference for Extracting Text Descriptors from Triage Images of Mass Disaster Victims

Earthquakes, hurricanes, terrorist attacks, and other such events can cause tremendous harm to people and infrastructure, often leaving those outside the areas impacted with little or no information on the state of their friends and relatives who may have been affected. Our goal is to automatically process images of patients taken as part of the intake process at emergencymedical care centers to extract searchable, text-based descriptors of patients that can be accessed remotely (e.g., via the web) to facilitate identification of victims. Lost Person Finder (LPF ) is a research project which aims at providing immediate post disaster communication information about disaster victims. As a part of the project, triage images of the victims are gathered and the aim is to extract a text descriptor that describes the person in the image. Achieving this goal requires finding the person in the image, and in particular their face, to drive the process of extracting features such as hair color and style, clothing color and style, the presence or absence of glasses, and other similar features. Face detection for such non–standard images is a difficult task and was the motivation behind starting work in this domain. As we dealt with each feature detector one at a time, we realized that the different biographical features are dependent on each other. The target is to exploit this inter– dependency between individual feature detectors and improve the efficiency of the complete system.